Practical application of the Delphi technique in a bicultural mental health nursing study in New Zealand

Background. Numerous studies have employed the Delphi technique to seek expert opinion about aspects of clinical practice. When researching literature on the Delphi technique, however, we discovered discrepancies in its application, and a lack of detail when reporting design, administration, and analysis methods. Such lack of specificity hinders the replicability and assessment of the clinical and cultural validity and reliability of Delphi studies.

Aim. The aim of this paper is to detail the practical application of the Delphi technique as a culturally and clinically valid means of accessing expert opinion on the importance of clinical criteria.

Methods. Reference is made to a bicultural New Zealand mental health nursing clinical indicator study that employed a three-round reactive Delphi survey. Equal proportions of Maori and non-Maori nurses (n = 20) and consumers (n = 10) rated the importance of 91 clinical indicator statements for the achievement of professional practice standards. Additional statements (n = 21) suggested by Delphi participants in round 1 were included in subsequent rounds. In round 2, participants explained the rating they applied to statements that had not reached consensus in round 1, and summarized responses were provided to participants in round 3. Consensus was considered to have been achieved if 85% of round 3 ratings lay within a 2-point bracket on the 5-point Likert-scale overall, or in one of the Maori nurse, non-Maori nurse, or consumer groups. A mean rating of 4·5 after round 3 was set as the importance threshold.

Findings. Consensus occurred overall on 75 statements, and within groups on another 24. Most statements (n = 86) reached the importance benchmark.

Conclusions. When rigorous methods of participant selection, group composition, participant feedback, and determination of consensus and importance are employed, the Delphi technique is a reliable, cost-effective means of obtaining and prioritizing experts judgements.

Transformation from student to occupational therapist : using the Delphi technique to identify the threshold concepts of occupational therapy

BACKGROUND/AIM: Understanding and facilitating the transformation from occupational therapy student to practitioner is central to the development of competent and work-ready graduates. However, the pivotal concepts and capabilities that need to be taught and learnt in occupational therapy are not necessarily explicit. The threshold concepts theory of teaching and learning proposes that every discipline has a set of transformational concepts that students must acquire in order to progress. As students acquire the threshold concepts, they develop a transformed way of understanding content related to their course of study which contributes to their developing expertise. The aim of this study was to identify the threshold concepts of occupational therapy. METHOD: The Delphi technique, a data collection method that aims to demonstrate consensus in relation to important questions, was used with three groups comprising final year occupational therapy students (n = 11), occupational therapy clinicians (n = 21) and academics teaching occupational therapy (n = 10) in Victoria, Australia. RESULTS: Participants reached consensus regarding 10 threshold concepts for the occupational therapy discipline. These are: understanding and applying the models and theories of occupational therapy; occupation; evidence-based practice; clinical reasoning; discipline specific skills and knowledge; practising in context; a client-centred approach; the occupational therapist role; reflective practice and; a holistic approach. CONCLUSION: The threshold concepts identified provide valuable information for the discipline. They can potentially inform the development of competencies for occupational therapy and provide guidance for teaching and learning activities to facilitate the transformation to competent practitioner.

European undergraduate curriculum in geriatric medicine developed using an international modified Delphi technique.

INTRODUCTION the rise in the number of older, frail adults necessitates that future doctors are adequately trained in the skills of geriatric medicine. Few countries have dedicated curricula in geriatric medicine at the undergraduate level. The aim of this project was to develop a consensus among geriatricians on a curriculum with the minimal requirements that a medical student should achieve by the end of medical school. METHODS a modified Delphi process was used. First, educational experts and geriatricians proposed a set of learning objectives based on a literature review. Second, three Delphi rounds involving a panel with 49 experts representing 29 countries affiliated to the European Union of Medical Specialists (UEMS) was used to gain consensus for a final curriculum. RESULTS the number of disagreements following Delphi Rounds 1 and 2 were 81 and 53, respectively. Complete agreement was reached following the third round. The final curriculum consisted of detailed objectives grouped under 10 overarching learning outcomes. DISCUSSION a consensus on the minimum requirements of geriatric learning objectives for medical students has been agreed by European geriatricians. Major efforts will be needed to implement these requirements, given the large variation in the quality of geriatric teaching in medical schools. This curriculum is a first step to help improve teaching of geriatrics in medical schools, and will also serve as a basis for advancing postgraduate training in geriatrics across Europe.

PIPc study: development of indicators of potentially inappropriate prescribing in children (PIPc) in primary care using a modified Delphi technique

Objective There is limited evidence regarding the quality of prescribing for children in primary care. Several prescribing criteria (indicators) have been developed to assess the appropriateness of prescribing in older and middle-aged adults but few are relevant to children. The objective of this study was to develop a set of prescribing indicators that can be applied to prescribing or dispensing data sets to determine the prevalence of potentially inappropriate prescribing in children (PIPc) in primary care settings.


Design Two-round modified Delphi consensus method.


Setting Irish and UK general practice.


Participants A project steering group consisting of academic and clinical general practitioners (GPs) and pharmacists was formed to develop a list of indicators from literature review and clinical expertise. 15 experts consisting of GPs, pharmacists and paediatricians from the Republic of Ireland and the UK formed the Delphi panel.


Results 47 indicators were reviewed by the project steering group and 16 were presented to the Delphi panel. In the first round of this exercise, consensus was achieved on nine of these indicators. Of the remaining seven indicators, two were removed following review of expert panel comments and discussion of the project steering group. The second round of the Delphi process focused on the remaining five indicators, which were amended based on first round feedback. Three indicators were accepted following the second round of the Delphi process and the remaining two indicators were removed. The final list consisted of 12 indicators categorised by respiratory system (n=6), gastrointestinal system (n=2), neurological system (n=2) and dermatological system (n=2).


Conclusions The PIPc indicators are a set of prescribing criteria developed for use in children in primary care in the absence of clinical information. The utility of these criteria will be tested in further studies using prescribing databases.

A Delphi study to validate an advanced practice nursing tool

Aim This paper is a report of a study conducted to validate an instrument for measuring advanced practice nursing role delineation in an international contemporary health service context using the Delphi technique. Background Although most countries now have clear definitions and competency standards for nurse practitioners, no such clarity exists for many advanced practice nurse roles, leaving healthcare providers uncertain whether their service needs can or should be met by an advanced practice nurse or a nurse practitioner. The validation of a tool depicting advanced practice nursing is essential for the appropriate deployment of advanced practice nurses. This paper is the second in a three-phase study to develop an operational framework for assigning advanced practice nursing roles. Method An expert panel was established to review the activities in the Strong Model of Advanced Practice Role Delineation tool. Using the Delphi technique, data were collected via an on-line survey through a series of iterative rounds in 2008. Feedback and statistical summaries of responses were distributed to the panel until the 75% consensus cut-off was obtained. Results After three rounds and modification of five activities, consensus was obtained for validation of the content of this tool. Conclusion The Strong Model of Advanced Practice Role Delineation tool is valid for depicting the dimensions of practice of the advanced practice role in an international contemporary health service context thereby having the potential to optimize the utilization of the advanced practice nursing workforce.

Major Sport Event Operational Planning Issues and Strategies: A Multi-Case Delphi Study

This study examined the operational planning, implementation and execution issues of major sport events, as well as the mitigation and management strategies used to address these issues, with the aim of determining best practices in sport event operational planning. The three Research Questions were: 1) What can previous major sport events provide to guide the operational management of future events? 2) What are the operational issues that arise in the planning and execution of a major sport event, how are they mitigated and what are the strategies used to deal with these issues? 3) What are the best practices for sport event operational planning and how can these practices aid future events? Data collection involved a modified Delphi technique that consisted of one round of in-depth interviews followed by two rounds of questionnaires. Both data collection and analysis were guided by an adaptation of the work of Parent, Rouillard & Leopkey (2011) with a focus on previously established issue and strategy categories. The results provided a list of Top 26 Prominent Issues and Top 17 Prominent Strategies with additional issue-strategy links that can be used to aid event managers producing future major sport events. The following issue categories emerged as having had the highest impact on previous major sport events that participants had managed: timing, funding and knowledge management. In addition, participants used strategies from the following categories most frequently: other, formalized agreements and communication.

Content validation for level of use of feature rich systems: a Delphi study of electronic medical records systems

Introduction. Feature usage is a pre-requisite to realising the benefits of investments in feature rich systems. We propose that conceptualising the dependent variable 'system use' as 'level of use' and specifying it as a formative construct has greater value for measuring the post-adoption use of feature rich systems. We then validate the content of the construct as a first step in developing a research instrument to measure it. The context of our study is the post-adoption use of electronic medical records (EMR) by primary care physicians. Method. Initially, a literature review of the empirical context defines the scope based on prior studies. Having identified core features from the literature, they are further refined with the help of experts in a consensus seeking process that follows the Delphi technique. Results.The methodology was successfully applied to EMRs, which were selected as an example of feature rich systems. A review of EMR usage and regulatory standards provided the feature input for the first round of the Delphi process. A panel of experts then reached consensus after four rounds, identifying ten task-based features that would be indicators of level of use. Conclusions. To study why some users deploy more advanced features than others, theories of post-adoption require a rich formative dependent variable that measures level of use. We have demonstrated that a context sensitive literature review followed by refinement through a consensus seeking process is a suitable methodology to validate the content of this dependent variable. This is the first step of instrument development prior to statistical confirmation with a larger sample.

Key stakeholders' expectations of educational outcomes from Australian critical care nursing courses: a Delphi study

Background
Educational preparation for critical care nursing in Australia varies considerably in terms of the level of qualification resulting in a lack of clarity for key stakeholders about student outcomes.

Objectives
The study aim was to identify and reach consensus regarding the desired learning outcomes from Australian post-registration critical care education programs as demonstrated through the graduate's knowledge, skills and attitudes.

Design
A Delphi technique was used to establish consensus between educators, managers, clinicians and students regarding learning outcomes expected of graduates with a Graduate Certificate, Graduate Diploma and Master level qualification in critical care nursing.

Participants
A total of 164 critical care nurses (66 clinicians, 48 educators, 32 managers and 18 students) participated and 99 questionnaires were returned in the first round (response rate 60%). Fifty-seven questionnaires were returned for Round 2 (response rate 58%).

Methods
Learning outcomes were obtained from the Australian College of Critical Care Nurses Competency Standards for Specialist Critical Care Nurses. Some statements included more than one characteristic, and these were split to create learning outcomes with one characteristic per item. A survey of Australian higher education providers of critical care education provided additional learning outcomes, for a total of 73 learning outcomes for the first Delphi round.

Results
Findings suggest that patient comfort, safety, professional responsibility and ethical conduct are deemed most important for all three levels of educational preparation. There was a lack of emphasis on clinical practice issues for all levels. Participants placed higher emphasis on learning outcomes related to complex decision-making, leadership, supervision, policy development and research for Graduate Diploma and Master level programs.

Conclusion
The findings have implications for curriculum development and the profession with regards to the level of educational preparation required of critical care nurses and suggest that further work is required before clear recommendations can be made for desired educational outcomes from critical care nursing programs in Australia.

Validação de constructo de tecnologia educativa para pacientes mediante aplicação da técnica Delphi

OBJETIVO: Descrever o processo de validação de uma tecnologia educativa para pacientes submetidos à cirurgia ortognática, mediante a aplicação da técnica Delphi. MÉTODOS: Participaram do estudo, dez juízes (quatro cirurgiões bucomaxilofaciais, duas enfermeiras, duas nutricionistas e duas fonoaudiólogas) que indicaram os assuntos a serem abordados no material destinado à primeira fase da técnica Delphi. Após a construção do material e a obtenção dos resultados da primeira fase, iniciou-se as segunda e terceira fases dessa técnica com validação da pertinência das informações contidas nessa tecnologia educativa. RESULTADOS: Na segunda rodada, obteve-se, pelo menos, 90,0% de concordância em todas as categorias. Na terceira, houve um aumento de 12% para o item conteúdo, melhora de 6,7% em linguagem, de 12,5% em ilustrações, de 3,3% no layout, de 10% em cultura, mantendo-se constante o item motivação. CONCLUSÃO: A técnica Delphi, pode ser uma importante ferramenta na construção e validação de tecnologias educativas.

Developments and prospects of freight railway transport in Northern Germany:a Delphi survey

Despite the fact that Germany has a well expanded traffic infrastructure, it confronts a strong growth in freight volumes and it is very likely that in the forthcoming years will not be able to reasonably cope with the increasing demand. The aim of this paper is to explore the developments of the transport sector, railways in particular, in North Germany. More specific, the paper explores whether the railway network could contribute to improved traffic development in North Germany and also whether or not the existing logistics practices are effective enough to cope with the traffic problems in the region. The research based on the Delphi technique, collected, analysed and summarised the opinions of a group of experts in the aforementioned issues. Results indicate that railways could represent the solution to the forecasted growing freight volumes in the next years. In spite of continuous efforts undertaken by the politics and the economy, the existing logistics and freight traffic concepts are not sufficient, financing is too scarce, while emerging issues like sustainability, environment protection and working conditions are taken into little consideration. Copyright © 2014 Inderscience Enterprises Ltd.

Efeitos do processo de acreditação nas práticas profissionais dos assistentes sociais em contexto hospitalar: perspectiva de um painel Delphi

A acreditação de um hospital revela-se um instrumento de extrema importância na melhoria da qualidade dos cuidados de saúde. Subjacente a este processo, estão inerentes alterações ao nível organizacional e individual. Este artigo tem como objetivo, identificar os efeitos do processo de acreditação nas práticas profissionais dos assistentes sociais em contexto hospitalar. Neste sentido, realizou-se um painel de peritos, com recurso à técnica Delphi, no qual participaram 12 peritos multidisciplinares e internacionais, tendo sido conseguida obtenção de consenso em três rondas. Dos resultados obtidos emergiram consensos que permite considerar cinco dimensões de ação relevantes face aos efeitos da acreditação nas práticas profissionais dos assistentes sociais, quanto: 1) especificidades da prática; 2) interação de equipas; 3) eficácia profissional; 4) avaliação e monitorização; 5) ganhos em saúde. São dimensões que correspondem a mudanças e que sugerem uma linha de ação que é influenciada, e determinada pelo processo de acreditação no desenvolvimento de uma cultura de melhoria contínua da qualidade da actividade do Serviço Social e do agir profissional. Os resultados obtidos evidenciam que um processo de acreditação tem efeitos diretos e positivos nas práticas dos assistentes sociais.

A comparative analysis of major ERP lifecycle implementation, management and support issues in Queensland Government

This paper reports on a study of ERP lifecycle major issues from the perspectives of individuals with substantial and diverse involvement with SAP Financials in Queensland Government. A survey was conducted of 117 ERP system project participants in five closely related state government agencies. A modified Delphi technique identified, rationalized and weighed perceived major issues in ongoing ERP life cycle implementation, management and support. The five agencies each implemented SAP Financials simultaneously using a common implementation partner. The three survey rounds of the Delphi technique, together with coding and synthesizing procedures, resulted in a set of 10 major issue categories with 38 sub-issues. Relative scores of issue importance are compared across government agencies, roles (client vs implementation partner) and organizational levels (strategic, technical and operational). Study findings confirm the importance of this finer partitioning of the data, and distinctions identified reflect the circumstances of ERP lifecycle implementation, management and support among the stakeholder groups. The study findings should also be of interest to stakeholders who seek to better understand the issues surrounding ERP systems and to better realise the benefits of ERP.

Emergency department surge capacity : Recommendations of the Australasian surge strategy working group

Background For more than a decade emergency medicine organizations have produced guidelines, training and leadership for disaster management. However to date, there have been limited guidelines for emergency physicians needing to provide a rapid response to a surge in demand. The aim of this study is to identify strategies which may guide surge management in the Emergency Department. Method A working group of individuals experienced in disaster medicine from the Australasian College for Emergency Medicine Disaster Medicine Subcommittee (the Australasian Surge Strategy Working Group) was established to undertake this work. The Working Group used a modified Delphi technique to examine response actions in surge situations. The Working Group identified underlying assumptions from epidemiological and empirical understanding and then identified remedial strategies from literature and from personal experience and collated these within domains of space, staff, supplies, and system operation. Findings These recommendations detail 22 potential actions available to an emergency physician working in the context of surge. The Working Group also provides detailed guidance on surge recognition, triage, patient flow through the emergency department and clinical goals and practices. Discussion These strategies provide guidance to emergency physicians confronting the challenges of a surge in demand. The paper also identifies areas that merit future research including the measurement of surge capacity, constraints to strategy implementation, validation of surge strategies and measurement of strategy impacts on throughput, cost, and quality of care.

An investigation of a process for environmental banking appropriate to Queensland

This thesis examines the advanced North American environmental mitigation schemes for their applicability to Queensland. Compensatory wetland mitigation banking, in particular, is concerned with in-perpetuity management and protection - the basic concerns of the Queensland public about its unique environment. The process has actively engaged the North American market and become a thriving industry that (for the most part) effectively designs, creates and builds (or enhances) environmental habitat. A methodology was designed to undertake a comprehensive review of the history, evolution and concepts of the North American wetland mitigation banking system - before and after the implementation of a significant new compensatory wetland mitigation banking regulation in 2008. The Delphi technique was then used to determine the principles and working components of wetland mitigation banking. Results were then applied to formulate a questionnaire to review Australian marketbased instruments (including offsetting policies) against these North American principles. Following this, two case studies established guiding principles for implementation based on two components of the North American wetland mitigation banking program. The subsequent outcomes confirmed that environmental banking is a workable concept in North America and that it is worth applying in Queensland. The majority of offsetting policies in Australia have adopted some principles of the North American mitigation programs. Examination reveals that however, they fail to provide adequate incentives for private landowners to participate because the essential trading mechanisms are not employed. Much can thus be learnt from the North American situation - where private enterprise has devised appropriate free market concepts. The consequent environmental banking process (as adapted from the North American programs) should be implemented in Queensland. It can then focus here on engaging the private sector, where the majority of naturally productive lands are managed.